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(|i) Improvements relating to control data arrays. 

(57) Apparatus for generating an array of control data, the array having a number of pixels each of which 
corresponds to a respective pixel in an image, the apparatus comprising a first store (100) for storing the 
image 

data ; a second store (17) for storing the control data ; and a processor which, for each control data 
array pixel generates and stores in the second store (17) a control value in accordance with a 
predetermined algorithm which defines the control value as a function of the corresponding image pael 
colour components and the corresponding colour component's) of the target colours, wherein the 
generated control values can be ordered on a scale of at least three value classes. 
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IMPROVEMENTS RELATING TO CONTROL DATA ARRAYS 

The invention relates to methods and apparatus for generating an array of control data, the array having 
a number pixels each of which corresponds to a respective pixel in an image. 

Such >ntroJ data arrays are commonly termed masks and fmd wide use in image processing systems. 
5 For example, in page layout they may define the areas and positions occupied by the features in the page, as 
well as the area of each image to be included. In image retouching, or colour correction they may define which 
pixels of an image are to be modified. In electronic paint systems they may represent the density of the spray 
from an air brush, in design systems they may indicate the density gradation of a vignette pattern or shadow 
region. 

10 Masks can be conveniently classified into two types : hard and soft. Hard masks have 1 bit per pixel. Their 

purpose is to separate the pixels of an image into two classes, one class to be processed in a certain way and 
the other to be processed in a different way. Soft masks, or "mattes", have multiple (usually 8) bits per pixel. 
They specify for each pixel a density value, which may represent the transparency (or opacity) of the pixel for 
a mixing operation. Into the latter category also fad soft-edged masks, having a hard interior but a graduated 

15 edge. These are typically used for "anti-aliased" compositing of curved shapes in computer graphics appli- 
cations. 

The use of masks simplifies both the operation and the software design of a product For the operator the 
advantages are that fewer items of information need to be kept in mind at any one time. He can concentrate 
first on making the mask, using a variety of electronic "tools" as appropriate, then on using it to control the scope 

20 of some operation, such as image retouching. Similarly the software designer can create one set of routines 
for making and manipulating masks, and another for the transforms on images or pages. 

The existence of a separate mask facilitates the design of hardware for the combination of two or more 
images for display on a video monitor. In the case of a hard mask, binary pixel values read from the mask store 
can be used to control a video-rate switch between the corresponding pixels of source images read synchrorv 

25 ously from separate Image stores. In the case of a soft mask, 8-bit pixel mask pixel values read from the mask 
store can be used to control a video-rate mixer that adds proportions of each of the source image pixels. Such 
arrangements are disclosed in EP-A-0089174 (equivalent US Patent 4,61 7,592) and EP-A-0344976 (equival- 
ent US Patent 495491 2). 

Masks can be generated conventionally in a number of ways. In the case of a hard mask, the operator might 
30 outline and fill an area of an image which is to be one class resulting in all the corresponding control pixels 
being coded with a binary "1 " while the remainder are coded with a binary "Cr. In another method, a colour selec- 
tive technique can be used to define a hard mask. In this technique each pixel of the source image is checked 
to determine whether the individual colour values (such as red, green, blue) fall into specified ranges. The 
true/false results are combined logically to give a final true or false (1 or 0) value for the corresponding mask 
35 pixel. The mask can then be displayed superimposed over the image. 

As an example, the algorithm used in the Crosfield Studio 800 system allows up to four classes of source 
colour to be defined. Each class consists of a range of values for one or more of the CMYK (cyan, magenta, 
yellow, black) inks (defaulting to 0-100% if not defined). A pixel of the source image is considered to be a mern^ 
ber of a class if all its colour components fall within the specified CMYK ranges (AND = logical intersection). 
40 A mask pixel is set if the corresponding image pixel lies in one or more of the classes (OR = logical union), 

i.e. if 

45 



The ranges C1 , M1 , etc are any arbitrary sets of values in each colour. Normally each range would repre- 
so sent one contiguous set of values, such as 40 ^ C 60, but non-contiguous sets are also allowed. 

Although the colour selective masking method described above gives usable colour selective masks, it suf- 
fers from several limitations : 

1 . The limits define a cuboid volume in colour space which is not always as selective as necessary ; 

2. It gives only a true/false selection, resulting in a binary or hard mask ; 

55 3. It does not perform well on translucent or reflective subjects where the background and foreground col- 

ours are mixed ; 
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4. The masks are affected by noise, resulting in unwanted "pin-holes" and ragged edges ; and 

5. It can be slow in implementation, resulting in frustrating delays for an operation that should ideally be 
interactive. 

In accordance with one aspect of the present invention, a method of generating an array of control data, 
5 the array having a number of pixels each of which corresponds to a respective pixel in an Image, comprises 
selecting at least one target colour, and then for each control data array pixel generating a control value in 
accordance with a predetermined algorithm which defines the control value as a function of the corresponding 
image pixel colour components and the corresponding colour components) of the target colour(s), wherein the 
generated control values can be ordered on a scale of at least three value classes. 
10 In accordance with a second aspect of the present invention, apparatus for generating an array of control 

data, the array having a number of pixels each of which corresponds to a respective pixel in an image, comprises 
a first store for storing the image data ; a second store for storing the control data ;and processing means which, 
for each control data array pixel generates and stores in the second store a control value in accordance with 
a predetermined algorithm which defines the control value as a function of the corresponding image pixel colour 
15 components and the corresponding colour components) of the target colours, wherein the generated control 
values can be ordered on a scale of at least three value classes. 

This invention improves upon the previous colour selective masking technique by obtaining a measure of 
the proximity of the colour coordinates of an image pixel to some target point in colour space, rather than simply 
making a binary decision about each colour component independently. 
20 In one example, the predetermined algorithm can define the control value as representing a function of the 

Euclidean distance in colour space between the image pixel colour and the target colour. For example, the pre- 
determined algorithm may have the form : 

D = V(a(C-Cr)2 + b(M-M T )2 ♦ c(Y-Y T )2 + d(K-K T )*) (1) 
where C T , Mr, Y T , and K T are the colour component values of the target colour, 
25 C. M, Y, K, are the colour component values of the image pixel, and 

a, b, c, and d are weighting coefficients. 

Although equation 1 defines the control value in terms of all four colour component values, this is not essen- 
tial and only one or some of the colour component values could be used. It should also be understood that any 
other suitable colour space could be used in place of CMYK, which is customary for printing applications. For 
30 example, in the frame stores for video display monitors pixels would typically be represented by red, green and 
blue colour components. In broadcast television, colour is encoded differently for transmission, such as the YIQ 
signals in the NTSC system. In yet other applications, such as paint and dye formulation, the CIE system is 
commonly used with LAB or LUV coordinates. The present invention applies without loss of generality to all of 
these, and other, colour spaces. 
35 For some purposes a simple distance metric, as described above, is not sufficient In order to generate a 

matte proportional to the "biueness" of the image, for example, a 2-D Gaussian distribution, centred on the target 
colour of blue might be more appropriate. This could be formulated as the product of two 1-D Gaussians : 

D = f(C,M) = aexp[-b(C-C T ) 2 ]expI-c(M-MT) 2 (2) 
where a is a normalising factor 
40 and b, c govern the widths of the distributions for C,M. 

Such functions can easily be extended to three or more colour coordinates as necessary. They are set out 
here in two colours only to facilitate visualisation. Instead of a Gaussian, an inverse power law might prove to 
be more useful having the general form : 

a 1 

45 D = g(C,M) = [C _ CJb + 1 [M _ Mt]c + 1 

One use for such functions could be in image retouching, for example to change all pixels of the target 
colour to another colour (e.g. blue to green) and to change pixels of nearby colours by a proportional amount 
The density matte value a at each pixel, normalised to the range [0-1 ] f could then be used to control the blending 
of the image with the new colour, for example by linear combination of each colour component : 

so Modified pixel = (1-a) x Old pixel + ax New colour (4) A further refinement of the technique can be made 

if the operator specifies not only the object (foreground) colour but also the background colour. The matte is 
then calculated in such a way that it takes high values for colours near the foreground target, low values for 
colours near the background target, and a medium value for all other colours. For example, using the Gaussian 
functions given in equation (2), the composite function could be : 

55 MCM) = fp(C r M) - f B (C,M) + f 0 (5) 

where f F is the foreground function 
f B is the background function 
and f 0 is a constant 
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The effect of this function is to have a maximum value at the foreground target colour, a minimum value 
at the background target colour and intermediate values elsewhere. 

In some cases the function may include a thresholding step in which an intermediate control value is com- 
part with a threshold to generate a final, binary control value or the final value may be generated from an 

5 inte. diate value and tone curves leading to a soft mask. 1 

3 target colour or colours themselves can be selected in a conventionaf manner by the operator simply 
specifying the colour component values for example by numerical entry via a keyboard. Alternatively, the 
operator can select a colour already existing in the image or mixed on a palette or can define the target colour 
as a range of related colours for example by using a pointing device such as a hand-held mouse to position a 

10 graphical cursor on the display screen over the desired colour(s). A preferred method for defining the target 
colour comprises specifying an area of interest within the image ; constructing a histogram of pixel values within 
the specified area of interest and determining from the histogram the median and given percentile points for 
each colour component, and then providing the target colour weighting and spread coefficients. 

In some circumstances, it has been found that selecting pixels with colours close to the target pixels can 

15 have surprising results. For example, an object which may appear generally red may actually contain a number 
of tints from blacks in deep shadows, intense reds where the object reflects internally, pinks and whites for high- 
lights, and so on. These tints are not necessarily the nearest to the target colour in the first colour space that 
is available, but span a range extending from the pure colour towards both white and black, 

We therefore propose an additional, preliminary, step in which the initial colour components defining the 

20 image in a first colour space are transformed into a second or subsequent colour space from which the control 
values can be determined. For example, in one case of particular interest the first colour space is transformed 
so that the colour components defining the central locus of the target colour range lie along the major axis of 
the second colour space, with the other two axes being perpendicular to the first axis and to each other. From 
this second colour space it is easy to derive the control values corresponding to each pixel of the image. 

25 The processing means may be implemented in a suitably programmed computer or in a hardware form. In 

the latter case, for example, in order to implement the algorithm defining equation (1) above, four look-up tables 
are provided each of which defines a function of the form X ( = (i-Y T )VN 2 

where Y T is a target colour component value and N is a scaling coefficient chosen so that the required 
amount of sensitivity for that colour component is achieved, outputs from the look-up tables being fed to adders 

30 which generate the sum of the outputs, the output from the adders being fed to a fifth look-up table which 
reduces the accuracy back to eight bits. This look-up table is commonly loaded with a linear or square root 
function. 

Other configurations are possible as will be described below. 

Some examples of methods and apparatus according to the invention will now be described with reference 
as to the accompanying drawings, in which :- 

Figure 1 illustrates graphically the relationship between a mask generated using a conventional colour 
selective technique and a mask generated by a method according to the invention ; 
Figure 2 illustrates the form of a weighted density function ; 

Figure 3 illustrates the form of a function which discriminates between foreground and background colour 
40 domains ; 

Figure 4 illustrates one example of a colour selective mask generation circuit ; 

Figure 5 illustrates a second example of a colour selective mask generation circuit ; 

Figures 6Aand6B illustrate an image and a colour component histogram respectively during the generation 

of a target colour ; 

45 Figures 7A-7E illustrate different stages in a method for preparing colour component data ; and, 

Figure 8 is a schematic block diagram of apparatus for performing the steps illustrated in Figure 7. 
Figure 1 illustrates a distance function in colour space, reduced to two-dimensions for simplicity, where 

the target colour has components C T and M T and an elliptical contour 1 results from calculating the distance 

function : 

so D = V(a(C-Cr)2 + b(M-M T )*) (6) 

The rectangular contour 2 which is shown illustrates, for comparison purposes, the effect of a conventional 
thresholding technique which simply makes a true/false decision depending upon whether or not C and M are 
within a range of values spanning the target colour. In this conventional case, any combination of C and M within 
the rectangle result in a mask value 1, anywhere outside the rectangle result in a mask value 0. 

55 In this example of the invention, the region in colour space can be restricted to an ellipse so that points 

that were included in the comers of the rectangle will now be excluded. The reduction in area is to fAB for the 
ellipse from 4AB for the rectangle, a constant ratio of f/4 = 0.79. In three dimensions the volume ratio of ellipsoid 
to cuboid is f/6 = 0.52. Thus the selectivity of the ellipsoid is almost double that of the cuboid. 

4 
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Figure 2 illustrates the form of the function defined by equation (2) above with the Z-axis representing the 
magnitude of the function. 

Figure 3 illustrates an example of the function defined in equation (5) above in two dimensions. 

Figure 4 illustrates a first example of a hardware arrangement for implementing the method, in this example, 
5 respective colour components C, M, Y and K of pixel data in the image store 100 are fed to four look-up tables 
3-6 (L1-L4), each having 256 addressable entries with an 8-bit data value stored at each entry. The outputs 
from the look-up tables 3, 4 are fed to an adder 7 while the outputs from the look-up tables 5, 6 are fed to an 
adder 8. The 9-bit outputs from the adders 7, 8 are fed to an adder 9 whose 10-bit output is used to address 
a look-up table 10 having 1024 entries, each containing an 8-bit mask value, which is stored in the correspond- 
to ing pixel location in mask store 17. 

In the simplest case, the look-up tables could be loaded as follows : 



15 
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20 

The scaling coefficient N is chosen so that the required amount of sensitivity is achieved. Out-of-range 
values are limited to 255. For example, N=1 concentrates attention on the narrow range C T ± 16 whereas N=8 
broadens the useful range to Cr ± 128. Differing values of N for each table give the ellipsoidal regions, i.e. dif- 
25 fering sensitivities for the different colour primaries. 

The multiplicative functions given by equations (2) and (3) can also be implemented by the circuit of Rgure 
4 by loading logarithms of the functions in the input tables and an antilogarithm in the output table : 



Ll - L4 X, = a . logff^i)) 

L5 Xf = exp(i/a) 



Figure 5 illustrates an alternative circuit, which is functionally almost identical to the circuit in Figure 4. In 
this case, the colour component data C, M and Y, K are fed in pairs from the image store 1 to look-up tables 

35 11, 12 (L10,L11) each of which has dimensions 64K x 8. The outputs from the look-up tables 11, 12 are fed to 
a further look-up table 1 3 (L1 2) to generate the final 8 bit value for storage in the mask store 1 7. The advantage 
of this arrangement is that the adders are omitted allowing more complex two-dimensional functions to be rep- 
resented (e.g. ellipses with rotated axes). 

In operation, the user will first define the target colour. This may involve, for example, displaying the image 

40 (Fig. 6A) on a monitor and then defining regions in the image having the colour which it is desired to mask. In 
this example, a colour-selective mask for the boy's T-shirt is required so that the operator might circle two reg- 
ions 14, 15 using a cursor or the like and then a further region 16 corresponding to an area which is not to be 
masked. 

The computer then "plots" all the pixels within the contours 14-16 in colour space and constructs a histog- 
45 ram for the distribution within each colour coordinate as illustrated in Fig. 6B. This histogram is then analysed 
to determine the median and given percentile points (in this case 10% and 90%) in each colour channel as 
shown in Fig 6B. The mean is used to define the colour component values of the target colour as shown. For 
example, the spread coefficients are derived from the statistical spread, or scatter, of the sample points as indi- 
cated for example by the 10%ile and 90%8e points of the histogram. They are then used to determine the length 
so of the semi-axes of the ellipse in Rgure 1, ie. the dimensions of the region in colour space that contains the 
majority of the desired colours. Equation (1) could be expressed as : 

(C-CrF (M-M T )2 (Y-Y T )2 

x* y* a* K) 

where 
55 x = - C t0 
y = Meo - C 10 
Z = Y90 - Y t Q 

The operator then instructs the computer to perform the appropriate algorithm as defined by the look-up 
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tables of either the Figure 4 or Figure 5 circuit The resultant mask is then stored in a store 17. To derive a 
hard (1-bit) mask for separating foreground and background regions of an image, the operator could subse- 
quently apply a simple threshold operation to the 8-bit data in mask store 1 7, displaying pixels of the mask where 
the value exceeds the threshold level by a distinctive colour such as bright green. Interactive adjustment of the 

5 threshold level would allow the operator to use judgement as to the optimum setting. 

As has been mentioned above, in some cases, it may be desirable to transform the colour components 
from one colour space to another before generating the colour-selective mask. The reason for this can be seen 
in Figure 7A where the probability contours of the range of colours in CMY colour space that can be seen in 
an object of a certain colour under different lighting conditions are shown in two dimensions. It can be seen 

10 that the central locus of these contours comprises a curve starting in the black comer, passing through the target 
colour, and then ending at the white point In the invention, we wish to plot the three-dimensional distribution 
of all colours within a certain degree of fit from the target colour, for example by sampling pixels from one or 
more areas of the image. It would be expected that this Figure w8l extend in the light/dark directions due to the 
large variety of lighting conditions, but will be quite confined in the perpendicular (hue) directions since hue is 

15 relatively unchanged with variations in lighting conditions. 

Ideally, one should transform the colour components to a second colour spaced that is based upon the 
perceptual colour attributes of lightness, colourfulness and hue (tCH), such as defined in the 1976 CIELUV 
Uniform Colour Space. Indeed, for applications where a high standard of colour fidelity must be maintained, 
such as image retouching and colour matching for print reproduction, use of such a colour space is essential. 

20 For making a mask, however, the quality criteria are less stringent and certain approximations can successfully 
be made. 

We have simplified the transformation by observing that the major diagonal in the CMY "colour cube" runs 
from black to white and is a reasonable approximation to the true achromatic lightness axis. In order therefore 
to improve the colour-selective masking technique, in an initial step the amounts by which the colour component 

25 values (CMY) defining the target colour must be modified to move those values so as to lie on the major diagonal 
in the colour cube (Figure 7B) are determined and then these weights are stored in an initial section of a set 
of nine look-up tables 50-58 (Figure 8). The initial CMY values are thus applied to respective sets of three of 
the look-up tables 50-58 which in the first section will convert these colour components to new colour values 
C\ M', Y', to define a set of elliptical contours aligned to the diagonal as shown in Figure 7B. The target point 

30 is now on the diagonal of the colour cube whOe the black and white points stay unchanged. 

The actual transformation is conveniently performed by scaling by linear amounts above and below the 
target colour, ie. by fitting straight lines from the target colour to the white and black points respectively. This 
introduces a geometric discontinuity at the target point, although in most instances this causes no visible dis- 
continuity in the final mask. This discontinuity could be eliminated by fitting a higher order curve, such as a 

35 parabola, through the target to the white and black points. 

I n the next stage the axes of the colour cube are rotated so that the first new axis (X') lies along the diagonal 
of the colour cube and the other two (Y\ Z') are perpendicular to it. This rotation results in the image being 
defined by new colour components X', Y\ Z' with contours as shown in Figure 7C. The rotation is performed 
by a second section of each of the LUTs 50-58 and a corresponding set of three summation circuits 59-61. 

40 Typically, each LUT 50-58 will also add an offset 

Although a complex transform, such as defined by the 1976 CIELUV formulae, would give the optimum 
results, in practice a simple linear combination of the CMY primaries gives acceptable results : 

45 X ' = (C+M+Y)/3 ) 

Y f = (M-C)/2 + offset ) (8) 

Z» = (2Y-C-M)/4 + offset ) 

so The X' axis approximates the lightness dimension in a perceptual colour space, and the Y\ Z' axes 

approximate the opponent red-green and yellow-blue dimensions respectively. 

In the next stage, the new axes are scaled by respective, different amounts, an origin shift is performed 
and the resulting component values are squared. This results in a series of circular contours (Figure 7D) and 
is performed by the look-up tables 62-64. The output values of these look-up tables 62-64 are reduced to eight 
55 bits and then fed to a summation circuit 65 and thence to an output look-up table 66 which may be loaded with 
a square-root or other function to derive the 8-bit mask value, in similar fashion to LUT 10 of Figure 4. 

Although the above description of the transformation implemented by Figure 8 has assumed the first colour 
space to represent the printing ink colours CMY, other colour spaces could equally well be catered for by minor 

6 
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same "colour cube" as described above. 



Claims 
1. 



10 



15 



can be ordered on a scale of at least three value classes. 

colour. 



3. A method according to claim 2. wherein the predetermined I algorithm has the form 
^ D = V(a(C-C T )2 + b(M-M T >2 + c(Y-Y T )2 + d(K-KT) 2 ) 0) 

where Cy. Mr. Y T . and K T are the colour component values of the target colour. 
C. M, Y. K, are the colour component values of the image pixel, and 
a, b. c. and d are weighting coefficients. 

2S 4. A method according to claim 1o,^ 

D = f(C,M) = a expI-b(C-CT) 2 ] exp^CM-Mr) 2 ] W 



where a is a normalising factor 
and b, c govern the widths of the distributions for CM. 



so 5 A method according to claim 1 or claim 2, wherein the predetermined algorithm has the form 
30 5. a memo y f^C,M) = f F (C,M) - f B (C,M) + f 0 (5) 



where 

f F is the foreground function 
f B is the background function 
35 and f 0 is a constant 
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45 



50 9 



55 



component, and then providing the target colour weighing and spread coeftaents. 

colour space from which the control values can be determined. 

with the other two axes being perpendicular to the first axe and to each other. 

ordered on a scale of at least three value classes. 
1 o. Apparatus according to claim 9. wherein the processing means is adapted to carry out a method according 
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to any of claims 1 to 8. 
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Fig. 5. 
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